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Preface 


Differential  game  models  of  realistic  problems  rarely 
lead  to  closed  loop  laws.  Major  G.  M,  Anderson  of  the 
Department  of  Mechanics  has  proposed  a means  of  approxi- 
mating  the  closed  loop  law  based  o si  updating  the  open  loop 
law.  This  thesis  is  a result  of  my  work  investigating  the 
applicability  of  his  method  to  the  air-to-air  missile 
intercept  problem  using  nonlinear  dynamics. 

I wish  to  acknowledge  the  assistance  of  Major  Anderson 
both  in  the  work  on  this  thesis  and  in  his  sequences  on 
Optimal  Control.  I would  also  like  to  thank  Squadron  Leader 
F.  D.  Reddyhof  for  his  assistance  with  the  missile  model 
and  the  proportional  navigation  scheme. 
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throughout  my  trials  and  tribulations. 
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Abstract 

Proportional  navigation  is  a closed  loop  optimal  control 
for  the  case  of  a linear  dynamic  model  of  the  air-to-air 
missile  intercept  problem  and  a quadratic  cost  function 
{Ref  6) . 

This  paper  presents  a differential  game  model  of  the 
intercept  problem  using  nonlinear  realistic  dynamics,  free 
final  time  and  a terminal  cost  function  related  to  probabil- 
ity of  kill.  With  this  model  proportional  navigation  is  no 
longer  optimal  and  the  extent  of  its  nonoptimality  is  indi- 
cated for  a range  of  saddle  point  solutions. 

A guidance  concept  based  on  differential  game  theory 
is  discussed  and  is  compared  to  proportional  navigation  in 
an  off  line  simulation.  The  considerable  gains  made  by  this 
scheme  over  proportional  navigation  provide  the  incentive 
to  develop  a real  time  version. 
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DIFFERENTIAL  GAME  GUIDANCE  VERSUS  PROPORTIONAL 
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i 

NAVIGATION  FOR  AN  AIR-TO-AIR  MISSILE 
I . Introduction 

Background 

Proportional  navigation  has  been  a major  guidance  law 

for  some  considerable  time  and  at  present  there  does  not  j 

| 

appear  to  be  any  viable  replacement  from  either  optimal 
control  theory  or  differential  game  theory.  Bryson  and  Ho 
(Ref  6:267)  show  that  proportional  navigation  is  an  optimum 
guidance  law  for  the  simple  case  of  linear  dynamics  and 
quadratic  cost  function  where  the  evader  is  not  maneuvering. 

Over  the  years  fighter  pilots  have  adopted  tactis,  based 
on  experience,  that  capitalize  on  the  shortcomings  of  pro- 
portional navigation. 

Attempts  have  been  made  to  devise  closed  loop  control 
laws  based  on  optimal  control  theory.  Two  such  examples 
(Refs  5 and  7)  use  linear  dynamics  and  a quadratic  cost 
function  and  produced  laws  that  require  knowledge  of  the 
future  tactics  of  the  evader.  However  no  allowance  is  made 
for  the  evaders  desire  to  capitalize  on  the  limitations  of 
the  missile  and  the  errors  in  its  prediction  of  the- 
evaders  controls. 

Using  differential  game  theory  it  is  possible  to  include 
in  the  optimal  control  problem  the  evaders  desire  and  pos- 
sible capacity  to  escape  interception.  In  theory,  therefore, 
an  optimum  guidance  law  could  be  devised  such  that  for  given 
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launch  conditions  the  probability  of  kill  was  some  minimum 
value  for  the  case  of  the  optimum  evader.  Any  nonoptimal 
tactics  on  the  part  of  the  evader  could  be  converted  into 
a higher  predicted  probability  of  kill  by  the  closed  loop 
optimal  control . 

The  optimal  controls,  for  a saddle  point  solution  of 
a differential  game  model  of  the  intercept  problem,  are 
defined  in  terms  of  a two  point  boundary  value  problem. 

For  nonlinear  dynamics  the  result  is  invariably  an  open 
loop  control  law  based  on  the  initial  states.  The  only 
means  of  updating  the  optimal  control  scheme  is  by  solving 
the  two  point  boundary  value  problem  at  each  instant  to 
account  for  the  nonoptimal  play  by  the  pursuer.  A near 
optimal  scheme  may  be  possible  by  linearizing  the  dynamics 
of  the  states  and  costates  about  the  nominal  saddle  point 
solution  and  periodically  updating  the  costates.  This  was 
proposed  by  Anderson  (Refs  1,  2,  and  4)  and  has  been  shown 
to  be  near-optimal  and  real  time  for  some  simple  problems 
where  closed  loop  laws  existed  for  comparison. 

The  principle  drawback  to  the  scheme  at  present  is  the 
computational  burden  in  that  numerical  integration  is  re- 
quired firstly  to  go  forward  to  the  predicted  final  range 
generating  the  nominal  trajectory  and  then  backwards  from 
final  time  generating  an  "update"  matrix  until  the  backward 
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date  is  made.  This  procedure  needs  to  be  repeated  often 
enough  to  keep  the  linearization  assumption  valid. 


Unfortunately,  for  a realistic  model  not  only  is  the  dimen- 
sion of  the  equations,  that  need  to  be  integrated,  quite 
high  but  also  the  amount  of  computation  at  each  step  is 
extensive . 

Statement  of  the  Problem 

This  scheme  is  not  capable  of  being  used  real  time  at 
present  on  a small  air-to-air  missile  but  could  be  used  in 
an  off  line  simulation  to  indicate  the  possible  performance 
of  such  a control  scheme  as  compared  with  proportional  navi- 
gation. 

The  aim  of  this  thesis  was  to  model  the  intercept 
problem  in  differential  game  theory  as  a zero  sum,  free 
final  time,  minimax  range  game  with  realistic  dynamics  and 
a cost  function  based  on  the  final  states.  The  saddle  point 
optimality  of  differential  games  is  valid  for  the  case  where 
at  least  one  player  has  a continuously  optimal  control.  Where 
both  players  play  their  open  loop  controls  this  condition  is 
met  but  where  either  one  deviates  the  open  loop  control  is 
no  longer  optimal.  By  means  of  a Fortran  subroutine  (Ref  10) 
that  solves  a set  of  n nonlinear  equations  in  n unknowns  a 
program  was  to  be  developed  to  provide  updates  to  the  open 
loop  controls  by  solving  the  two  point  boundary  value  problem 
at  intervals  throughout  the  interception.  The  continuously 
optimal  controls  of  either  pursuer  or  evader  could  then  be 
approximated  for  the  purposes  ■>{  evaluating  the  performance 
of  proportional  navigation. 


The  following  cases  could  then  be  studied: 

(a)  Optimal  pursuer  against  optimal  evader 

(b)  Proportional  navigation  against  the  open  loop 
optimal  evader  of  (a) 

(c)  Proportional  navigation  against  updated 
optimal  evader  of  (a) 

(d)  Updated  optimal  pursuer  against  nonoptimal 

evader 


(e)  Proportional  navigation  against  the  nonoptimal 
evader  in  (d)  . 


Summary 

The  air-to-air  missile  interception  of  an  aircraft  is 
modelled  as  a two  person,  zero  sum,  minimax  range  differen- 
tial game.  Chapter  II  presents  a brief  discussion  of  game 
theory.  Chapter  III  presents  the  model  used  for  the  inter- 
cept problem,  based  on  a typical  infrared  seeking  missile 
and  a F4  type  aircraft,  and  derives  the  optimal  controls 
in  terms  of  the  states  and  costates  together  with  the  two 
point  boundary  value  problem.  Chapter  III  also  presents  the 
model  used  for  proportional  navigation.  The  results  are 
presented  in  Chapter  IV  with  examples  of  each  of  the  cases 


discussed  above. 


1 1 . Different  is  1 Game  Theory 


Problem  Formu 1 at  ion 

The  mathematical  elements  of  differential  game  theory 
comprise  the  state  equations,  a cost  function,  control  or 
path  constraints  and  a terminal  surface.  The  vector  differ- 
ential equation  describing  the  motion  of  the  two  vehicles  is 


x = f(x,u,v,t)  x(tQ)  = (2-1) 

where  u is  the  control  vector  of  the  pursuer  and  v the  con- 
trol vector  of  the  evader.  The  cost  function  in  its  general 
form  is 


J 


$ (x,t) 


(2-2) 


For  a zero-sum  game  the  pursuer  attempts  to  minimize  J 
whilst  the  evader  attempts  to  maximize  J,  using  their  respec- 
tive control  vectors.  These  controls  may  be  subject  to 
inequality  constraints  of  the  form 


C(xp,u)  < 0 

C (x  , v)  < 0 
6 


(2-3) 


where  x , x are  the  components  of  the  state  vector 
P e 

dated  with  the  puisuer's  and  the  evader's  position 
velocity , 

The  game  being  considered  is  a free  final  time 
with  no  terminal  constraints  and  "minimax"  range  as 


x asso- 
and 

problem 
the  cost 
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function.  The  "rainimax"  range  cost  function  may  include 
terms  associated  with  the  velocity  vector. 

An  augmented  cost  function  is  formed 


J 


J 


[ AT  ( f -x  ) + ;.C]  dt 


(2-4) 


where 

A = costate  vector 
p = Lagrange  multiplier  vector 
» 0 for  C < 0 
t 0 for  C = 0 

A saddle  point  solution  is  presumed  to  exist  such  that 
J(u* ,v)  < J(u*,v*)  < J(u,v*)  (2-5) 


where 

u*,v*  = optimal  controls 

u,v  = any  other  admissible  control 
The  necessary  conditions  for  the  saddle  point  solution 
to  the  game  for  a Hamiltonian  that  is  separable  in  u and  v, 
where  the  Hamiltonian,  H,  is  given  by 

H s ATf  + L = H (x , v ) + H (x.u)  (2-6) 

e p 

is  as  follows  (Ref  6:277) 


H 


*T(tf)  = ♦. 


I 


1 


6 


i 
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H = 0 
u 


Hy  - 0 


H(tf)  = -*t 


(2-7) 


These  conditions  are  for  interior  controls  and  should  the 
controls  be  constrained  then  the  conditions  become 


A * -H  - yC 
x x 


A‘(tf)  - *x(tf) 


H = -yC 
u u 


H = -yC 
v v 


H(tf)  = -*t 


(2-8) 


These  conditions  together  with  the  state  dynamics  are  a 
two  point  boundary  value  problem  (TPBVP)  of  the  form 


x = f(x,A,t)  x(tQ)  = 


A - g(x,A,t)  A (t -)  = 4>, 


H(tf)  = -♦t 


t^  free 


(2-9) 


The  solution  of  this  TPBVP  provides  A(tQ),  such 
that  the  necessary  conditions  are  met.  The  controls  that 
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need  to  be  applied  to  optimize  the  cost  function  can  be 
found  from 


X - f(x,\,t) 

x(t0)  * x0 

> = g(x, *,t) 

X{*(P  “ xo 

(2-10) 

and  by  integrating  these  state  and  costate  equations  forward 
to  tf  the  trajectories  of  the  saddle  point  solution  to  the 
game  are  given. 

The  controls  defined  in  Eq  (2-10)  are  valid  for  the  case 
where  both  the  players  use  the  saddle  point  controls.  Should 
one  player  use  nonoptimal  controls  then  the  value  of  the 
cost  function  may  change  in  his  opponent's  favor  provided  his 
opponent  can  adjust  his  controls  to  capitalize  on  the  non- 
optimal play. 

Where  the  optimal  controls  are  determined  from  Eq  (2-10) 
in  the  form 

u ( t ) = u(x0,AQ,t) 

v(t)  = v(xQ,X0,t)  (2-11) 

the  controls  are  known  as  open  loop  controls  and  will  not 
automatically  adapt  to  changes  in  the  game.  In  order  to 
capitalize  on  nonoptimal  play  the  values  of  X(tj)  must  be 
amended  by  some  amount  <5 A ( t j ) based  on  the  difference  be- 
tween the  value  of  x(t,)  from  the  saddle  point  solution  and 

A 

the  actual  value  of  x(t^)  as  a result  of  some  nonoptimal 
play. 


8 


Pseudo  Closed-Loop  Scheme 


In  some  neighborhood  of  the  saddle  point  trajectory 
the  expressions  of  Eq  (2-10)  can  be  expanded  about  this 
nominal  trajectory  ignoring  terms  of  second  order  and 
higher  to  give 

Sx  = fx(x,X,t)5x  v fx(x,X.t)6X 

6X  = gx(xJXJt)6x  + gx(xJX,t)<SX  (2-12) 

By  similar  reasoning  the  transversal ity  conditions  can 
be  considered  as  linear  functions  of  fix(t^),  6X(t^)  and  dtf. 
Anderson  in  Ref  1 shows  that  for  any  set  of  transversal ity 
conditions  the  small  change  in  the  costates  can  be  repre- 
sented by 


6X(tf)  = A ' x ( t f ) (2-13) 

The  matrix  A is  evaluated  at  the  nominal  terminal 
states,  costates  and  final  time. 

The  corresponding  change  in  the  state  at  some  prior 
time,  tj,  due  to  small  changes  in  the  terminal  states  can 
be  expressed  as 

5x ( 1 1 ) = ♦jx(ti ,tf) 6x(tf)  + ^xX(tl ’t{^SX ttf)  (2-14) 

where 

A , 4 , are  state  transition  matrices. 

XX  xX 

The  changes  in  the  costates  are  then 

«X(tj)  = + ^,\X(tl  •tf^6X(tf)  (2-15) 
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Hence  for  a small  deviation  from  an  open  loop  optimal 
trajectory  due  to  sub-optimal  play  on  the  part  of  one  player 
the  change  Sxtt^)  can  be  used  to  predict  a change  in  costates 
dA(tj)  that  updates  the  costates,  and  hence  the  open  loop 
controls,  to  account  for  thu  small  divergence  from  the 
previous  saddle  point  solution. 

Thu  relationship  between  and  dx(tj)  is  found  by 

integrating  back  from  the  expected  terminal  state  either  a 
matrix  ricatti  equation  or  the  time  derivatives  of  the  transi- 
tion matrices  of  Eq  (2-14]  and  similarly  for  the  costates. 

This  integration  backwards  from  the  expected  final  time  is 
terminated  as  it  reaches  the  current  real  time  state.  The 
difference  between  the  states  at  that  time  if  both  players 
had  played  optimally  and  the  actual  states  provides  the 
vector  fixtt^).  This  small  change  in  state  is  then  used  to 
update  the  costates  to  convert  the  suboptimal  play  of  one 
player  into  a gain  for  the  other. 

Anderson  (Refs  1,  2,  and  4)  has  proposed  such  a means 
of  updating  the  saddle  point  solution  to  capitalize  on  non- 
optimal  play  and  the  references  give  comparisons  with  closed 
loop  laws  for  some  simple  problems.  The  updating  is  done 
in  real  time  and  is  shown  to  be  near  optimal. 
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III.  I n t c r c c jvt  Problem 

The  particular  game  considered  is  a zero  sum  differen- 
tial game  between  an  air-to-air  missile  and  an  aircraft 
with  final  time  free. 

Game  Scenario 

The  game  scenario  is  as  follows: 

(a)  The  aircraft  model  is  based  on  the  F4,  with 
altitude  and  velocity  dependent  stall  limits,  thrust  and 
drag  . 

(b)  The  missile  is  based  on  typical  boost-coast 
air-to-air  missile  with  an  infra-red  seeker.  The  missile 
guidance  scheme  begins  at  the  end  of  the  boost  phase  and, 
as  for  the  aircraft,  has  altitude  and  velocity  dependent 

drag . 

(c)  The  vehicles  are  represented  as  point  masses 
flying  co-ordinated  turns  in  3 dimensions. 

(d)  Gravity  is  include^  but  taken  to  be  constant 
in  direction  and  magnitude. 

(e)  Final  time  is  left  free. 

(f)  The  game  ends  when  (J)  - 0. 

Aircraft  Model 

The  F4  has  been  the  basis  for  the  aircraft  model  with 
polynomials  used  to  represent  the  variations  in  stall  limits 
and  maximum  thrust  with  velocity  and  altitude.  The  vehicle 


1 1 


dynamics  are 


*#v 


V 


9 £ 03  V COS  0 

y » v eos  y sin  a 
% k v sin  v 
v - [T  cos  ct  - D) 

Y * [ L f T sin  a ] 


I 

— - g s 5.  n y 

m ' 


cos  P . g cos  Y 
JEV  V 


° s l L 


T sin  « } 


si n 

my  cos  v 


(3-1} 


where 

x - distance  north 
y = distance  west 
z - height 

v = velocity  magnitude 

Y - angle  velocity  vector  makes  with  the 
horizontal 

a = angle  between  the  velocity  vector  projected 
into  the  horizontal  plane  and  the  x axis, 
a = angle  of  attack 

a £ angle  between  thrust  and  velocity  vectors 
0 = bank  ang  1 e 
D = drag  force 

L = lift  perpendicular  to  wing  plan  form  and 
perpendicular  to  velocity  vector 
g = gravity 

T = thrust,  along  a/c  centerline  . 


The  controls  are  load  factor  and  bank  angle.  Bank 
angle  is  unconstrained  and  for  this  problem  max  throttle 


has  been  specified,  with  thrust  then  dependent  upon  alti- 
tude and  speed.  The  constraints  on  the  load  factor  are 
taken  to  be  a function  of  altitude  and  speed  below  the 
corner  velocity  and  a structural  limit  above  the  corner 
speed  of  6 g's. 

Finally  the  drag  force  is  taken  to  be 


0 = CD  QS 


where 
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kC, 
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S = reference  area 


(3-2) 


Missile  Model 

The  missile  model  is  similar  to  the  aircraft  model  but 
has  the  following  differences. 

(a)  Thrust  = 0 

(b)  No  aerodynamic  constraint  on  load  factor. 

The  missile  is  considered  in  the  coast  phase  only. 

The  load  factor  constraint  is  taken  at  15  g's  independent 
of  speed  and  altitude. 

Cost  Function 

The  cost  function  for  the  game,  which  the  evader  wishes 
to  maximize  and  the  pursuer  to  minimize,  is 

J = AR2  - B cos  (TCA)  (3-3) 
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where 


R = distance  between  the  a/c  and  the  missile 

TCA  = angle  between  th-ir  respective  velocity 

vectors  (Track  Cros? .ng  Angle) 

A,B  = positive  weighting  factors 

2 

The  justification  for  using  "R  '*  is  that,  although  it 
produces  large  values  of  some  of  the  costates  at  t„  through 
the  transversal ity  conditions,  the  effectiveness  of  the 
missile  warhead  will  probably  be  inversely  proportional 
to  R . The  justification  for  using  "-cos  (TCA)"  is  that 
the  cost  function  should  in  some  way  represent  the  problems 
associated  with  fuzing  a warhead.  By  and  large,  to  achieve 
a high  probability  of  kill,  a missile  should  fly  a parallel 
flight  path  with  the  a/c  at  the  final  time.  The  term  in 
TCA  represents  a minimum  cost  of  -B  for  a tail-on  chase  in 
the  final  stages.  The  cost  varies  slightly  for  small  values 
of  TCA  but  increases  rapidly  for  angles  above  45<>.  It  will 
also  penalize  the  pursuer  for  a head-on  attack  on  the  basis 
that  the  relative  range  rate  is  very  high,  which  makes 
fuzing  very  difficult. 

At  a later  stage,  when  investigating  the  behavior  of 
the  game  where  the  missile  had  more  than  sufficient  control 
necessary  to  intercept  the  aircraft,  the  cost  function  was 
changed  slightly.  In  order  to  provide  a cost  function  that 
was  canable  of  rtirprri  no  the  missile  to  choose  between  non- 

•A  Q 

unique  optimal  controls,  a small  penalty  related  to  final 
time  was  added.  Its  magnitude  was  such  that  if  the  missile 
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could  not  intercept  the  aircraft  at  all,  or  if  the  missile 
could  intercept  the  aircraft  but  not  at  an  acceptable  track 
crossing  angle  then  the  time  penalty  was  insignificant. 
However  for  situations  where  the  missile  could  bring  the 
original  cost  function  to  its  absolute  minima  then  the  time 
penalty  became  significant. 

The  cost  function  used  was  therefore 

J = AR2  - B cos  (TCA)  + C tf  (3-4) 


with 

A = 1 
B = 10 
C = 1/6 

These  values  were  chosen  to  make  the  TCA  term  signifi- 
cant below  10  ft  and  the  t^  term  significant  near  the  minimum 
value  of  the  TCA  term  when  R(t^)  ->0. 


D i f f er ent i a 1 Game  Guidance 

By  application  of  the  necessary  conditions  for  a saddle 
point  solution  the  controls  can  be  expressed  as  functions  of 
the  states  and  costates. 

The  costate  dynamics  are  found  from  the  partial  deriva- 
tive of  the  Hamiltonian  with  respect  to  tht  corresponding 
state  e . g . 
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(3-5) 


producing  the  following 


15 


MMMlHiMii 


MiMMWMiWWiMMMiMMiMI 


ffissastM 


X = A = 0 
x y 


(3-6) 


X = 
z 


■H 

■[ 


3T  T _ 3a  3D 

- cos  a - T sin  « ^ 


1 


X cos  u A Sin  w 


mv 


mv  cos  y 


[ 


— + •—  sin  a + T cos 
3z  3z 


3a  *1 

a 3z  J 


(3-7) 
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X [-v  sin  y cos  c] 

v 1 


A [-v  sin  y sin  c] 
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X [v  cos  y] 


X v [ - g COS  y] 
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(3-9) 
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X = -X  f-v  cos  y sin  o] 
ox1  J 


•Ay[v  cos  y cos  o] 


The  controls  used  are  n,  the  load  factor  and  y the  bank 
angle.  The  evader's  thrust  is  set  to  max  throttle  and  is 
then  state  dependent  while  the  pursuer's  thrust  is  zero. 


with 


Hence  the  unexpanded  partial  derivatives  above  become 
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(3-17) 
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= 6Cl 


The  load  factor  control  is  constrained  and  so  account 
must  be  taken  of  the  effect  of  the  constraint  on  the  costate 
equation . 


Cj (n, z ,v)  > 0 


C2(n)  > 0 


where  Cj  is  the  constraint  below  the  corner  speed  and  C2  the 
constraint  above  the  corner  speed  and. 


Cj  = n^z.v)  - n > 0 


C2  = 6-  n>0 


Where  the  control  is  constrained 
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(3-28) 
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Hence 


X = - H - H C 
z z n z 


X = - H - H C 
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(3-29) 

(3-30) 


for  constrained  load  factor  below  the  corner  speed  and  since 
the  constraint  above  the  corner  speed  is  state  independent 
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(3-31) 


\ « - «v  (3-32) 

as  before.  In  the  case  of  the  pursuer  the  missile  is  con- 
sidered to  have  sufficient  control  at  all  speeds  of  interest 
that  only  the  state  independent  constraint  is  in  force. 

For  numerical  purposes  a Fortran  integer  variable  "mu" 
was  used  to  switch  the  additional  terms  on  and  off  as  neces- 
sary in  the  costate  equations  so  that  as  the  load  factor 
selected  by  the  control  algorithm  was  constrained  to  the 
state  dependent  limit  "mu"  was  set  to  1,  otherwise  it  was 
left  at  0. 


Optimal  Contro?  s 

For  an  unconstrained  control  such  as  the  bank  angle 
the  first  order  necessary  conditions  for  an  optimal  control 
require  that 
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The  second  order  necessary  conditions  require  that 


H <0 

uv 


for  a maximizing  player.  Combining  these  two  equations  we 
are  able  to  define 
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For  the  minimizing  player  we  find 
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In  the  case  of  the  load  factors  we  solve  for  the  uncon- 
strained load  factor  that  is  a stationary  control  of  the 
Haniltonian.  Should  there  be  an  interior  control  that  opti- 
mizes the  Hamiltonian  we  check  the  sufficient  conditions  to 
ensure  maximization  or  minimization  as  the  case  may  be. 

Where  however  there  is  no  interior  control  that  is  a station 
ary  control  with  respect  to  H we  choose  either  maximum  or 
minimum  load  factor  to  achieve  constrained  optimization. 

For  the  evader  then  we  consider  the  first  order  neces- 
sary conditions 
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Substituting  for  cos  u etc  and  making  small  angle  approxima- 
tions 
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For  an  interior  control  the  second  order  conditions  are 
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i.e.,  Xy  > 0 for  n,  above,  to  be  a maximizing  control 


22 


For  a constrained  load  factor  the  choice  is  either  max 
or  sir.  load  factor.  The  Hamiltonian  terms  that  contain 
load  factor  dependent  terms  are 


H,  = — (T  cos  a - D)  + — (L  ♦ T sin  a)  cos  y 
i a av 


♦ £ L + T sin  a)  sin  y 
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For  A > 0 we  can  look  for  an  interior  control  and  if 
v 

n so  defined  is  too  large  we  choose  n = n max.  For  A <0 

v 

we  find  that  our  sufficient  condition  predicts  a minimiza- 
tion of  H in  some  region  of  negative  load  factor.  In  order 


to  naximize  the  Hamiltonian  we  select  n * n max  for  Ay  < 0, 
The  exact  opposite  arguments  are  used  to  select  the 
optimum  load  factor  for  the  missile.  Thus,  for  interior 
controls 
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which  requires  A <0  for  an  interior  control  to  be  possible. 
VP 

Sufficient  conditions  for  a minimum  with  respect  to  the 
Hamiltonian  are 
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(3-53) 
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Hence  X^  < 0 for  minima  and  following  similar  arguments 

for  the  pursuer  as  for  the  evader  we  choose  the  constrained 
load  factor  to  be 


and 


n = n for  X > 0 
max  v 


(3-54) 


n = n for  X <0 
max  v 


(3-55) 


Proportional  Navigation  Scheme 

In  order  to  check  the  validity  of  the  results  a propor- 
tional navigation  algorithm  was  produced  for  the  missile  and 
used  against  an  open  loop  optimal  evasion  scheme. 

We  expected  to  find  that  the  proportional  navigation 
scheme  (PN)  would  fJy  a similar  course  to  the  open  loop 
optimal  missile  but  would  not  get  as  close  to  the  evader. 

For  the  purposes  of  the  test  the  PN  scheme  was  allowed 
to  use  the  exact  current  values  of  the  vehicle  velocities 
and  positions  to  measure  the  spin  rates  of  the  line  of 
sight.  The  line  of  sight  was  defined  by  two  angles,  one 
an  azimuth  angle  and  the  other  the  angle  out  of  the  hori- 
zontal plane.  For  the  Cartesian  frame  centered  on  the 
missile  as  in  Fig.  1 the  two  angles  are  respectively  e,  41. 


Fig.  1.  Lin*  of  Sight  Angles. 
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We  wish  to  fly  the  missile  so  that  y , and  a are 

P P 

some  multiple  of  p and  8. 
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and  then 
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(3-63) 


If  n so  defined  exceeded  IS  it  was  set  at  15. 

These  values  of  y,  n were  then  used  in  Eq  (3-60)  and 
Eq  (3-61)  to  give  the  time  rate  of  change  of  y and  o for 
the  missile. 

Tn  the  case  where  6 is  very  small  or  zero  the  evalua- 
tion of  the  load  factor  in  Eq  (3-63)  leads  to  numerical 
inaccuracies.  In  this  case  Eq  (3-60)  is  used 
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IV.  Results 


TPBVP  Solution  Method 

The  results  can  be  divided  into  two  phases  namely 

(a)  nominal  saddle  point  solutions 

(b)  variations  about  the  nominal  solution. 
Subroutine  NSOIA  was  utilized  in  both  phases.  Firstly,  it 
was  used  to  provide  a one  dimensional  search  for  a point 
on  the  terminal  surface,  and  secondly  it  was  used  to  solve 
two  point  boundary  value  problems,  both  in  the  refining  of 
the  nominal  saddle  point  solution  so  as  to  minimize  the 
errors  resulting  from  numerical  integration  and  in  the  up- 
dating schemes  to  solve  the  two  point  boundary  value  prob- 
lem (TPBVP)  at  each  update  point.  Appendix  B presents  a 
short  discussion  on  the  principles  of  the  subroutine  and 
on  an  aspect  of  the  numerical  integration  of  the  state  and 
costate  equations  that  is  the  probable  cause  of  variations 
in  a Hamiltonian  which  should  be  constant. 

Saddle  Point  Solutions 

In  order  to  produce  a large  number  of  saddle  point 
solutions  covering  a wide  range  of  terminal  conditions, 
points  on  the  terminal  surface  were  found  by  selecting 
eleven  out  of  twelve  of  the  terminal  states  and  varying 
the  remaining  state  until  the  transversality  condition  on 
H(t^)  was  met.  Reference  3 provides  justification  for  this 
approach  since  it  is  shown  that  barriers  do  not  occur  in 
the  game  formulation. 
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By  integrating  backwards  from  the  terminal  surface  it 
was  possible  to  construct  saddle  point  solutions  to  games 
which  ended  at  30,000,  20,000  ar.d  10,000  ft  with  final  miss 
distances  from  0 to  200  ft  and  with  a variety  of  track 
crossing  angles  (TCA) . TCA  was  defined  as  the  angle  b-  tween 
the  missile  and  aircraft  velocity  vectors  at  t^.  By  inte- 
grating forward  again  to  it  was  found  that  there  were 
errors,  due  to  the  numerical  integration  scheme,  and  the 
transversality  conditions  were  not  met  exactly.  These 
could  be  reduced  by  letting  NSOIA  vary  the  initial  costates 
slightly  until  these  errors  in  the  transversality  conditions 
were  reduced  to  some  acceptable  level. 

The  measure  of  convergence  for  the  TPBVP  was  the  sum 
of  the  squares  of  these  errors,  namely 
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Because  of  the  dynamics  and  the  cost  function  it  was 

possible  to  reduce  the  TPBVP  to  eleven  states  by  removing 

1 and  X . Hence  for  a value  of  e = 121  the  average  error 

xp  yp 

in  ..he  tr  ans  ver  sa  i i ' y conditions  would  be  1.  With  the  pro - 
g i' am  coded  in  ft,  it/sec  etc.  this  represents  an  error  of 


1 ft  in  the  final  range.  For  values  of  e less  than  100  the 
trajectories  showed  little  change  as  z was  decreased  and  it 


was  possible  to  keep  N'SOIA  searching  until  e was  less  than 
.001  at  which  point  the  terminal  miss  distance  was  fixed 
to  within  a fraction  of  an  inc.i.  e was  therefore  used  to 
define  acceptable  convergence  in  the  solution  of  TPBVP's. 

For  refining  nominal  solutions  e was  set  at  20  and  for  the 
updating  scheme  e was  set  at  400  so  as  to  reduce  computer 
time.  In  a typical  updating  scheme  approximately  25  TPBVP's 
were  solved  and  it  was  necessary  to  sacrifice  some  accuracy 
to  keep  total  running  time  to  a minimum. 

Open  Loop  Controls 

With  final  time  free  and  with  the  cost  function  used 
the  open  loop  optimal  trajectories  looked  realistic.  The 
optimal  evader  normally  turns  across  the  missile's  path  and 
in  most  instances  descends.  The  aircraft  was  kept  subsonic 
because  polynomials  representing  load  factor  limit,  maximum 
thrust  and  drag  as  functions  of  height  and  velocity  were 
only  valid  subsonic.  As  a result  the  aircraft  was  usually 
below  its  corner  speed  in  the  cases  considered  and  by 
descending  it  was  able  to  turn  faster.  The  aircraft  also 
gained  thrust  and  load  limit  advantages  by  descending. 

Both  missile  and  aircraft  used  maximum  load  factor  along 
smooth  arcs  to  the  terminal  surface.  This  was,  for  nonzero 
final  range  , the  point  at  which  each  had  the  same  radius  of 
turn  and  the  aircraft  was  on  the  inside  of  the  missile. 

The  missile  paid  a large  drag  penalty  using  maximum  load 
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factor  ar.d  would  typically  give  up  1500  ft/sec  in  velocity 
during  the  6 second  engagement, 

The  initial  relative  position  of  the  two  vehicles 
6 seconds  away  from  termination  varied  from  a beam  attack 
at  5000  ft  range  for  a final  range  of  0.  ft  to  12,000  ft 
with  the  missile  about  40®  off  the  front  of  the  aircraft 
for  final  ranges  of  200  ft. 

In  all  the  cases  studied  interior  controls  were  never 
used  and  despite  attempts  to  force  interior  controls  by 
utilizing  zero  final  range  and  a TCA  of  zero  and  forcing 
interior  controls  at  when  integrating  backwards,  interior 
control  solutions  were  never  found. 

The  justification  for  free  final  time  is  not  only  that 
it  is  more  realistic  but  also  for  terminal  pay  off  it  is 
essential  for  valid  results  as  the  following  example  shows. 
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It  appears  that  in 

about  2 secs  at  most  the  missile  will  achieve  a maximum 
hit.  The  same  problem  under  a free  final  time  formulation 
is  shown  in  Figs.  2 anu  5.  The  3D  trajectory  is  given  in 
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Fig.  2 with  the  ground  tracks  of  the  two  vehicles  plotted 
on  the  bottom  of  the  box  containing  the  trajectories.  The 
bank  angle  selected  in  the  saddle  point  solution  is  plotted 
along  with  the  proportional  navigation  (PN)  choice  of  bank 
angle.  The  PN  plot  represents  the  choice  of  bank  angle 
based  on  line  of  sight  (LOS)  considerations  as  the  missile 
flies  along  the  optimal  trajectory.  The  aircraft  flies 
across  the  path  of  the  missile  and  dives.  The  missile  is 
then  forced  into  a maneuver  whereby  it  begins  to  fly  to 
the  right  of  the  initial  position  of  the  evader  and  then 
continues  to  climb  above  the  evader  for  an  approach  with 
the  minimum  TCA  it  can  achieve.  Both  vehicles  use  maximum 
load  factor  throughout  the  5.108  secs. 

It  appears  that  the  fixed  final  tine  problem  guarantees 
the  aircraft's  safety  provided  RCt^.)  is  nonzero.  It  also 
does  not  prevent  range  going  to  zero  at  some  intermediate 
stage.  The  trajectory  in  Fig.  2 has  a smaller  final  range 
than  that  of  Ref  9 in  similar  times  and  yet  ultimately  the 
result  of  Ref  9 will  be  more  in  the  missile's  favor  than 
that  of  Fig.  2. 

A selection  of  open  loop  saddle  point  trajectories  are 
shown  in  Figs.  4 through  15.  For  comparison  purposes  plots 
are  made  of  the  bank  angle  selected  by  the  missile  on  the 
basis  of  optimal  control  and  proportional  navigation.  Since 
the  missile  flies  along  only  one  of  these  paths  the  bank 
angle  selection  of  the  alternate  scheme  is  not  accurate. 

In  the  very  early  stages  the  inaccuracies  will  be  small  and 
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The  variations  of  optimal  control  (OPT) , nonoptimal 
control  (NONOPT) , and  proportional  navigation  (PN)  considered 
are : 

(a)  saddle  point  solution 

(b)  open  loop  OPT  evasion  versus  PN  pursuit 

(c)  updated  OPT  evasion  votsus  PN  pursuit 

(d)  updated  OPT  pursuit  versus  NONOPT  evasion 

(e)  PN  pursuit  versus  NONOPT  evasion. 

The  open  loop  optimal  evader  results  demonstrated  that 
for  all  but  a few  cases  the  open  loop  optimal  evader  gained 
against  proportional  navigation.  The  few  cases  were  the 
problems  where  the  saddle  point  solution  lay  in  a plane  of 
constant  X or  Y and  hence  bank  angle  was  predetermined. 

Since  PN  used  maximum  load  factor  in  these  cases  the  out- 
come was  the  same  as  for  the  saddle  point  solution.  Figures 
12  and  13  are  two  such  cases.  Figures  16  through  23  present 
two  cases  for  comparison  of  open  loop  optimal  control  versus 
proportional  navigation  for  an  open  loop  optimal  evader. 

In  each  case  the  open  loop  optimal  evader  gains  against 
proportional  navigation.  In  this  group  of  figures  the  bank 
angle  selection  of  the  two  schemes  when  flown  along  the 
proportional  navigation  path  is  given  after  the  proportional 
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navigation  pursuit  trajectory  plot.  Plots  are  also  given 
of  the  rotation  rates  of  the  line  of  sight  (LOS)  whilst 
guided  by  the  proportional  navigation  scheme. 

Tables  II  and  III  give  the  initial  conditions  for  the 
two  cases  presented  in  Figs.  16  through  23.  Table  I gives 
the  final  ranges  for  the  two  cases  for  the  optimal  control 
and  proportional  control  missile  against  open  loop  optimal 
evader . 


Table  I 

Fina 1 Ranges  for  OPT  and  PN 
Fig.  No.  Final  Range 


16 

17 

20 

22 


182 

440 

1.76 

11 


Tuned  Proportional  Navigation 

An  incidental  result,  on  which  little  time  was  spent, 
is  mentioned  here  because  it  greatly  improved  the  perfor- 
mance of  the  PN  scheme.  For  a given  set  of  initial  condi- 
tions and  for  an  evader  playing  the  open  loop  optimal  control 
the  PN  scheme  could  be  "tuned"  to  improve  its  performance. 

The  two  navigation  gain  constants  for  PN  had  previously  been 
set  to  the  same  value.  By  introducing  a ratio  between  these 
gain  constants  considerable  improvement  in  the  performance 
was  achieved. 
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The  case  considered  was  the  problem  in  Fig,  16,  Tne 
PN  scheme  used  maximum  load  factor  throughout  the  flight 
and  only  the  bank  angle  differed  between  PN  and  OPT  guidanc  . 
The  bank  angle  selected  by  PN  had  similar  characteristics 
and  so  the  initial  bank  angle  of  the  PN  scheme  was  brought 
closer  to  the  initial  bank  angle  fot*  the  OPT  guidance.  The 
means  of  varying  the  PN  bank  angle  was  to  select  some  ratio 
for  RK1:RK2,  the  vertical  and  horizontal  gain  constants. 

For  a ratio  1:2  the  final  range  had  been  brought  down  from 
440  feet  to  207  feet  which  compared  very  favorably  with  the 
OPT  scheme  result  of  182  feet. 

An  example  of  the  advantages  of  the  tuned  navigation 
scheme  is  given  in  Figs.  24  through  27.  The  ratio  used  for 
RK1 : RK2  was  1:2  and  this  ratio  was  not  optimized.  It  was 
merely  a first  estimate  of  an  improvement  and  since  ratios 
of  1:1.7  and  1:2.3  gave  similar  improvements  1:2  was  used 
to  present  the  results.  Figures  24  and  25  are  the  open  loop 
saddle  point  solution  with  the  "tuned"  proportional  naviga- 
tion bank  angle  plotted  whilst  flying  the  optimal  trajectory. 
Normal  proportional  navigation  achieved  a 478  ft  miss  and 
with  "tuned"  proportional  navigation  the  result  was  191  ft 
against  the  173  ft  for  the  saddle  point  solution.  Table  IV 
presents  the  initial  conditions. 

Pseudo  Closed  Loop  Guidance 

For  an  open  loop  solution  to  be  useful  the  evader  must 
play  optimally  or  the  open  loop  solution  must  be  updated. 
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The  open  loop  scheme  was  updated  at  approximately  .3  sec 
intervals  throughout  the  attack  by  solving  the  associated 
two  point  boundary  value  problem.  The  solution  was  considered 
acceptable  when  the  sum  of  the  squares  of  the  errors  in  the 
transversality  conditions  was  less  than  400.  The  new  solu- 
tion could  then  be  used  to  update  the  optimal  missile  attack- 
ing a nonoptimal  evader  ot  an  optimal  evader  escaping  a PN 
missile.  This  was  an  expensive  process  in  terms  of  computer 
resources  and  is  completely  out  of  the  question  for  real  time 
guidance.  It  does  however  provide  valuable  information  as 
to  the  magnitude  of  the  changes  in  the  costates  and  demon- 
strates what  gain  could  be  made  in  the  “cost"  for  whichever 
player  was  being  updated. 

The  motivation  for  this  approach  is  that  a proposed 
real  time  pseudo- c losed  loop  scheme  (Ref  1,  2,  and  4)  will 
probably  have  to  give  up  some  of  the  realism  of  the  model 
for  the  advantages  of  a better  behaved  two  point  boundary 
value  problem.  In  order  to  test  simpler  models  for  their 
ability  to  provide  sufficient  advantages  over  proportional 
navigation,  or  in  fact  any  other  scheme,  to  warrant  imple- 
mentation, then  some  means  of  comparison  is  required.  The 
work  of  this  thesis  is  therefore  intended  to  provide  a ref- 
erence performance  standard.  Table  V and  Fig.  20  provide 
a saddle  point  solution  as  the  basis  for  the  various  compari- 
sons that  can  be  made.  These  are: 

(a)  open  loop  optimal  evasion  versus  proportional 


{b}  updated  optimal  evasion  versus  proportional 

navigation 

(c)  updated  optimal  pursuit  versus  nonoptimal 

evasion 

(d)  proportional  navigation  versus  nonoptimal 

evasion. 

These  cases  are  presented  in  the  same  order  in  the  following 
sections . 

Open  Loop  Opt ima  1 Evasion 

The  saddle  point  optimality  condition  for  differential 
games  requires  that  at  least  one  of  the  players  uses  an 
optimal  control.  The  open  loop  solution  for  the  evader 
against  propovt j onal  navigation  has  neither  player  playing 
optimally.  However  the  result  does  indicate  that  the  evasive 
maneuver  capitalizes  on  the  nonoptimality  of  proportional 
navigation.  v.'-g’?.res  30  through  32  present  the  trajectories, 
the  bank  angle  oc  the  proportional  navigation  missile  and 
the  optimal  gulfed  missile  whilst  flying  the  proportional 
navigation  trajectory  and  the  line  of  sight  rates  resulting 
from  the  proportional  navigation  scheme.  The  result  is  a 
20  ft  miss  witn  a TCA  of  1.1  radians. 

Updated  Optimal  Evader 

Where  the  optimal  evader,  pursued  by  the  proportional 


navigation  missile,  is  allowed  to  update  its  controls  at 
intervals  to  capitalize  on  the  nonoptimal  play  of  the 
pursuer  considerable  gains  are  made.  The  predicted  final 


range,  track  crossing  angle  and  the  cost  function  assuming 
the  missile  reverts  to  optimal  control  given  by  the  TPBVP 
solution  at  each  update  are  given  in  Table  VI.  The  range 
R is  in  feet  and  the  track  crossing  angle  TCA  is  in  radians. 
The  cost  function  and  the  degree  of  convergence  are  also 
given.  t is  the  predicted  time  to  go,  again  assuming  both 
players  play  optimally,  and  the  column  tQ  - t gives  a compar- 
ison with  the  column  r to  show  how  the  game  is  extended  by 
the  updating  optimal  evader  to  gain  further  advantage. 

The  entry  at  line  22  in  the  table  is  the  final  range  (180  ft). 
Figures  33  and  34  present  the  results  for  the  updated  optimal 
evader.  The  bank  angle  selected  by  the  optimal  missile  is 
actually  perturbed  by  being  forced  along  the  PN  trajectory 
in  Fig.  34.  It  does  show,  however,  that  the  PN  scheme  is 
near  optimal  and  that  the  solution  to  the  TPBVP  at  the  up- 
date points  was  realistic. 

For  the  case  of  the  updated  optimal  evader  the  costates 
are  presented  in  tabular  form  in  Tables  VIII  and  IX.  At 
each  update  point  the  current  values  of  the  costates  are 
given  in  the  "in"  column.  The  values  of  the  costates  after 
the  return  from  NS01A  are  given  in  column  "out".  The  game 
is  then  integrated  forward  to  the  next  update  point  and  the 
current  values  of  the  costates  are  used  as  guesses  and  appear 
in  the  "in"  column.  The  change  in  costates  between  updates 
is  therefore  the  difference  between  the  costates  in  the  "out" 
column  at  one  update  point  and  the  "in"  column  at  the  next 
update . 
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The  close  similarity  between  proportional  navigation 
bank  angle  and  optimal  control  bank  angle  in  Fig,  34  sug- 
gests that  "tuned"  proportional  navigation  may  well  have 
made  considerable  improvements.  Unfortunately,  no  time  was 
available  to  consider  this  case. 

Updat ed  Pursuer 

These  cases  are  those  where  the  missile  is  allowed  an 
update  at  .3  second  intervals  against  a nonoptimal  evader. 

The  evader  flies  max  g load  at  a bank  angle  of  90°  as  a 
nonoptimal  evasion. 

Flying  optimally  the  evader  could  only  achieve  a final 
range  of  1.5  feet.  The  final  time  was  6.004  secs.  During 
the  updating  computer  run  a less  accurate  solution  was  taken 
to  be  acceptable.  The  first  call  to  N301A  solves  the  initial 
two  point  boundary  value  problem  and  the  time  to  go  was  set 
at  6 sec.  The  answer  was  acceptable  within  the  new  accuracy 
requirements  but  the  small  change  in  time  alone  of  4/1000 
secs  changed  the  terminal  miss  to  4.4  feet  with  a sum  of 
squares  of  the  transversal ity  errors  at  240.  The  results  in 
Table  V represent  a solution  with  an  error  of  14. 

Table  IX  gives  the  results  from  the  computer  run.  The 
overriding  factor  in  determining  the  number  of  updates,  the 
acceptable  accuracy  and  the  maximum  allowable  number  of 
iterations  in  the  solution  of  the  TPBVP  was  the  central 
memory  time  requirements.  These  results  are  therefore  not 
ideal  in  that  the  trade  offs  between  accuracy  and  central 


memory  tine  have  not  allowed  convergence  in  many  instances. 
The  instability  cf  the  resulting  controls  reflect  this  lack 
of  convergence  but  at  intervals  the  solution  converges  and 
the  nissile  promptly  makes  up  lost  ground.  However,  in  the 
last  time  period  the  missile  is  flying  open  loop  against  a 
nonoptiaal  evader  who  is  able  to  double  his  miss  distance 
during  this  last  critical  stage. 

An  interesting  aspect  was  the  fact  that  at  updates  12, 
13,  14  the  11th  update  open  loop  solution  remained  accept- 
able. With  no  update  made  the  evader  gained  ground  against 
the  missile.  Figures  35  and  36  represent  the  trajectory  and 
the  optimal  bank  angle  respectively. 


Proportional  Navigation  with  Nonoptimal  Evasion 


In  this  case  the  nonoptimal  evasion  maneuver  is  used 
against  a proportional  navigation  missile.  The  result  is  a 
20  ft  miss  with  a TCA  of  2.02  radians  which  is  more  in  the 


evader's  favor  than  the  open  loop  optimal  evasion.  Where 
neither  player  plays  a game  optimal  strategy  the  saddle  point 
optimality  is  not  meaningful.  It  does  suggest  that  only  a 
poor  estimate  of  the  open  loop  evasion  strategy  makes 
reasonable  gains  against  proportional  navigation.  The  up- 
dated evader  however  makes  considerable  gains  for  this 
problem.  That  the  updated  optimal  evader  cannot  always 
make  such  startling  gains  can  be  seen  in  Tables  X and  XI. 
These  two  problems  are  based  on  saddle  point  solutions  that 
and  in  similar  ranges  but  different  TCA's.  In  the  first 
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case  the  saddle  point  solution  is  given  in  Table  III  and 
Fig.  20.  The  bank  angle  plot  of  Fig.  21  indicates  the  near- 
optimal  nature  of  proportional  navigation.  As  a consequence 
the  updated  evader  can  only  extend  the  range  to  22  ft. 
However  for  the  case  of  Table  XI  the  updated  evader  makes 
considerable  gains  against  proportional  navigation.  The 
saddle  point  solution  is  given  by  Fig.  6.  The  1.73  ft  of 
the  saddle  point  solution  is  converted  to  370  ft  by  the  up- 
dating evader. 

Summary 

The  problems  associated  with  the  solution  of  the  TP8VP 
for  very  small  final  ranges  is  reflected  in  the  instability 
of  the  updating  scheme  for  the  pursuer.  Where  he  has  an 
open  loop  final  range  that  is  small,  noncptinal  play  on  the 
part  of  the  evader  is  difficult  to  convert  to  a decrease  in 
the  payoff.  Where  the  saddle  point  solution  has  a much 
larger  final  range,  initially,  while  the  pursuer  is  convert- 
ing nonoptimal  play  by  the  evader  into  a decrease  in  the 
payoff,  the  TPBVP  is  amenable  to  updating  within  the  number 
of  iterations  allocated.  However  as  the  final  range  gets 
smaller  the  TPBVP  becomes  more  awkward.  In  one  case  the 
pursuer  converted  a 240  ft  final  range  to  .7  ft  at  some 
intermediate  stage.  The  updating  scheme  was  able  to  keep 
the  final  predicted  range  to  about  2 ft  despite  Iurt  of 
convergence  at  some  update  points.  However  in  the  last 


phase  under  open  loop  control  the  predicted  final  range  of 
2 ft  became  11  ft. 

The  updating  evader  has  this  aspect  in  his  favor  since 
he  is  continually  increasing  the  final  range.  The  problem 
becomes  more  well  behaved  and  hence  easier  for  bin.  to  update. 
F.ven  if  he  doesn’t  update  the  open  loop  saddle  point  strategy 
provides  seme  small  gain  against  proportional  navigation 
missiles. 

In  the  cases  considered  no  interior  controls  were  found 
even  for  proportional  navigation.  This  is  probably  because 
for  an  open  loop  optimal  evasion  or  even  for  a rough  approx- 
imation to  optimal  evasion  the  missile  is  forced  into  maximum 
g maneuvers.  Hence  proportional  navigation  loses  because 
of  its  inability  to  lead  or  lag  its  bank  angle  based  on 
considerations  other  than  the  immediate  line  of  sight  rate. 
Many  cases  in  this  thesis  show  bank  angle  patterns  very  simi- 
lar to  optimal  control  with  only  a constant  offset.  In 
other  instances.  Fig.  3 for  example,  there  is  considerable 
difference  in  the  dynamic  behavior,  even  though  at  a later 
stage  the  bank  angle  selections  are  essentially  the  same. 

The  gain  constants  for  the  PN  scheme  were  normally  10. 

This  represents  a performance  for  PM  beyond  the  current 
capabil ities . 
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V.  Conclusions  and  Recommendations  • 
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Conclusions  ; 

This  thesis  compares  the  principle  of  proportional 
navigation  to  differential  game  optimal  control.  But  for 
a few  cases  presented  where  launch  conditions  are  very 
favorable  to  the  missile,  proportional  navigation  normally 
pays  a considerable  penalty,  in  terms  of  the  cost  function, 
in  that  the  missile  has  the  energy  and  the  maneuverability 
in  greatly  improve  the  probability  of  kill  under  optimal 
control.  The  realistic  dynamics  used  in  this  thesis  are 
unlikely  to  provide  the  basis  for  a pseudo-closed  loop 
optimal  control  scheme  as  suggested  in  Refs  1,  2,  and  4. 

However  the  considerable  ains  that  can  b made  suggest 
that  simpler  models  may  be  used  so  as  to  ompromise  between 
gains  in  the  payoff  and  in  achieving  a real  time  capability. 

I t appears  that  the  optimal  evader  has  most  to  gain 
using  this  scheme  since  the  numerical  problems  improve  as 
the  final  range  increases.  The  open  loop  control  is  effec- 
tive against  proportional  navigation  so  that  any  failure 
to  update  the  control  does  not  relinquish  existing  gains. 

Whereas  any  failure  to  update  the  missile  control  increases 
the  final  range. 

Lastly  I conclude  that  the  subroutine  NS01A  is  a power- 
ful means  of  solving  the  TPBVP't  associated  with  saddle 
point  solutions.  Considerable  care  should  be  used  in 
selecting  the  parameters  that  control  NSOlA's  performance 
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and  in  this  respect  NS01A  remains  in  the  same  class  as  j 

! 

most  other  schemes  in  that  a good  understanding  of  the  j 

| 

nature  of  the  solution  is  essential  to  obtaining  a solu-  j 

tion,  no  matter  how  powerful  a numerical  method  is  used.  ! 

Re commend at  ions 

It  is  recommended  that  further  work  in  this  subject 
should  concentrate  on  finding  simpler  game  models  that 
behave  in  the  same  characteristic  manner  as  the  nonlinear 
model  of  this  thesis.  Since  interior  controls  have  not 
been  found  it  may  be  possible  to  fix  load  factor  at  its 
maximum  value  and  reduce  the  problem  to  a single  uncon- 
strained control  in  bank  angle  for  each  player.  Under 
these  conditions  a simpler  drag  model  may  well  prove  suf- 
ficiently accurate.  If  a simpler  model  produces  similar 
behavior  then  a pseudo-closed  loop  real  time  optimal 
guidance  scheme  may  well  prove  to  be  possible  for  simula- 
tion purposes  if  not  eventually  for  missile  guidance. 
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Table  II 


Sadd le  Point  Solution 


State 

Evader 

Pursuer 

X 

4972  ft 

9199  ft 

y 

2986  ft 

10260  ft 

z 

33150  ft 

33170  ft 

V 

706  ft/sec 

2219  f t / s 1 
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Sadd  1 e Point  Solution 
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Table  IV 


Saddle  Point  Solution 


State 

Evad  er 

Pursuer 

X 

4936  ft 

9341  ft 

y 

3013  ft 

1035S  ft 
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33198  ft 

33326  ft 
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7 03  f t / s ec 

2235  ft/sec 
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-.84  rads 

- . 62  rads 
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Point 

Solution 

Pursuer 

tQ  = o 

tf  = 6 

C+ 

O 

II 

o 

r+ 

It 

x ft 

4178 

8003 

10920 

8002 

y ft 

4435 

1999 

10242 

1998 

z ft 
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10003 

7782 
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v ft/sec 
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Y rad  s 

- . 5 

. 1 

- . 29 

1 . 08 

a rads 

-1.2 

.0 
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A 

X 
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Table  VI 


"OPT"  Evasion 


R 

TCA 

Cost 

Error 

T 

To  - 

0 

4.39 

1 . 14 

15 

239 

6 

6 

1 

31.45 

1.10 

985 

2767 

5.74 

5.707 

2 

58.12 

1 . 05 

3374 

16541 

5 . 49 

5.414 

3 

56 

1 . 00 

3131 

129 

5 . 24 

5.121 

4 

68.9 

. 97 

4747 

196 

4 . 97 

4.828 

5 

79.5 

. 94 
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259 

4 .70 
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6 

88 . 3 
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304 
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95 . 5 
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4.16 
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Evader’s  Costate  History 
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12 
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67 

63 

77 
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90 
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67 
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77 

89 
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98 
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80 

98 

■ or 

8 

136 

148 

80 

81 
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148 

160 

81 
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115 
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17  1 
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13 
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6596 

42452 

11881 

75319 

2 

-48 

- 136 

42496 

134076 

71511 

223344 

3 

-104 

-140 

132856 

217525 

210976 

341921 

4 

- S5 

-87 
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6 

30 

66 
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1 
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7 

106 
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320259 
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8 
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221 

301699 

3 l 8 1 29 

378058 

394004 
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243 

286 

2964  2 1 

308454 
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366781 

10 
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3 3 4 1 46 
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343 
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15 

403 
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135257 

16 
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123596 

1 17  086 
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95047 

17 

346 
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91235 

7742  1 

7 1922 

41846 

18 

302 

326 

57999 

4 207  6 

30068 
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19 

233 

241 

29536 

15  017 

339 

-24454 

20 
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9261 

1491 

-13174 

-26901 

21 
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39 
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R(tf} 

TCA 

Cost 

Error 

,:o  ' 

0 

4 . 4 

1.14 

151 

239 

6 

6 

i 

27 

1.22 

780 

4931 

5 . 64 

.5.7  07 

2 

44 

1.4 

2003 

11645 

5 . 2 

5.414 

3 

8.6 

1 . 55 

73 

37  52 

4 . 8 

5.121 

4 

17.4 

1 . 55 

303 

2086 

4.41 

4 . 828 

S 

1.19 

1 .49 

.64 

2109 

4 , 07 

4 . 535 

6 

4.18 

1 . 46 

16.49 

1995 

3.72 

4.242 

7 

1.40 

1 . 6 

2.67 

825 

3.17 

3 . 949 

8 

2.S 

1 . 87 

9 . 26 

254 

2 .69 

3 . 656 

9 

.95 

2.01 

5.17 

65 

2.31 

3.363 

10 

9.04 

2.01 

8S.9 

390 

2 .02 

3 . 070 

11 

6.942 

2.06 

52.9 

3 7 6 

1.70 

2.777 

12 

4 . 8 

2 . 06 

27.9 

195 

1.41 

2.484 

13 

1.91 

2 . 06 

8.39 

36 

1.12 

2.191 

14 

5.66 

2 . 06 

36 . 8 

191 

.828 

1 . 598 

15 

6.19 

2. 07 

43 . 2 

202 

.53  2 
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16 
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220 
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17 
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Table  X 


"OPT”  Evasion 


R 

TCA 

Cost 

Error 

T 

To  * 

0 

1 • 7 S 

.086 

-6.85 

13 

6 

6 

1 

11.8 

.04 

131 

2205 

5.77 

5.765 

2 

12.7 

.06 

151 

1357 

5.56 

5.531 

3 

10.0 

.06 

SO 

295 

5.38 

5.296 

4 

11.6 

.05 

124 

105 

5 . 2 

5.063 

5 

15.5 

. 05 

232 

421 

4.99 

4.828 

6 

15.0 

.05 

216 

176 

4.81 

4.593 

7 

15.9 

.04 

245 

184 

4.61 

4.359 

8 

16.9 

.04 

277 

141 

4.41 

4.125 

9 

17.2 

.05 

287 

112 

4.21 

3.891 

10 

17.5 

.05 

297 

141 

4.00 

3.656 

11 

17.6 

.05 

302 

219 

3.78 

3.422 

12 

17.7 

.06 

304 

15 

3.56 

3 . 188 

13 

17.7 

.05 

304 

249 

3.32 

2.953 

14 

17.7 

. 06 

304 

136 

3.09 

2.719 

15 

17.8 

. 06 

307 

6 

2.36 

2.484 

16 

18.0 

. 06 

315 

291 

2.62 

2 . 250 

17 

18.3 

. 06 

3 256 

17 

2.39 

2.016 

18 

18.7 

. 06 

340 

68 

2.16 

1 .781 

19 

19. 1 

. 06 

?53 

200 

1.92 

1.547 

20 

19.3 

. 07 

362 

257 

1.69 

1.313 

21 

19.8 

. 07 

382 

265 

1 . 46 

1.078 

22 

19.4 

. 07 

368 

176 

1 . 23 

.844 

23 

19.8 

. 07 

381 

242 

1.00 

.609 

24 

21 . 

. 07 

432 

287 

.76 

.375 

25 

21.4 

. 07 

448 

28  1 

.52 

.141 

26 

21.9 

. 06 

472 

235 

.2'  9 

- .094 
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R 
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Cost 

Error 

T 

tg  ‘ 

0 

1.73 

1.12 

-1.3 

. 5 

6 

6 

1 

86.8 

1 . 06 

7 529 

34742 

5.82 

5.765 

2 

164 

1 . 03 

27  127 

104577 

5 ,65 

5.531 

3 

5.2  5 

. 968 

15677 

329 

5 . 53 

5 . 296 

4 

1 S 9 

.917 

2 S 4 5 5 

369 

5.37 

5.063 

5 

190 

.868 

" 5282 

30 

5 . 20 

4.828 

6 

218 

.820 

47  57  9 

86 

5 .02 

4.593 

*» 

242 

.775 

5 8 8 8 ! 

84 

4 . 83 

4 .359 

8 

264.2 

.732 

698  19 

148 

4 . 64 

4.125 

9 

283 

. 693 

80106 

297 

4 .44 

3.891 

10 

299 

.657 

89522 

75 

4 . 23 

3.656 

1 1 

312 

.625 

97  91  1 

130 

4 . 02 

3.422 

12 

324 

. 598 

105  179 

67 

3 .80 

3 .188 

13 

333 

. 577 

111295 

260 

3 . 56 

2.953 

14 

341 

. 56 

116291 

120 

3.31 

2.719 

15 

346 

. 55 

12024? 

8 

3 . 07 

2 .484 

16 

35  1 

. 54 

1 23256 

1 17 

2.82 

2.250 

17 

354 

. 54 

1^5443 

28 

2 . 57 

2.016 

18 

356 

. 53 

1 26920 

189 

2.32 

1.781 

19 

357 

. 53 

1 27806 

172 

2 . 07 

1.547 

20 

358 

. 53 

1 28  23  1 

157 

1.82 

1.313 

21 

358 

. 53 

128350 

43 

1 .55 

1.078 

22 

358 

.53 

128362 

7 

1 .35 

.844 

23 

358 

. S3 

1 28540 

1 04 

1.12 

.609 

24 

359 

.52 

1 2924  2 

25 

.91 

.37  5 

25 

361 

.51 

130932 

1 

. 69 

.141 

26 

366 

. 48 

1 33954 

3 9 

. 48 

- .094 

27 

37  1 

.45 

' * 7 8 7 1 
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25 
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Append  i x 8 
Kuiserica  1 A s p e c t.  s 

TPBVP  Solution  Method 

A saddle  point,  solution  to  a differential  game  for  - u- 
latiors  is  defined  by  n two  point  boundary  value  problem  in 
the  fora 


a = f (x  , X , t) 
i - g (x  , , t) 
t ^ -•  free 


x (tc-)  = x 0 


(tfl 


H(t  ) - -♦  ! 


( B - 1 } 


where  there  are  n+1  unknowns  at  t0>  namely  i(tQ)  and  and 
at  some  time  t ^ there  are  n+1  relationships  that  must  hold. 
The  problem  could  be  more  generally  stated  as  follows 


F (x)  - 0 


(B  -21 


where  x •-  estimates  of  n+1  unknowns  in  E q s (B-l)  and  the 
vector  function  F represents  the  relationships  that  must 
hold  to  some  acceptable  degree  of  accuracy  at  time  t^.,  The 
solution  of  Eq  (B-2)  provides  the  appropriate  unknowns  at. 
time  t q such  that  if  b o t h players  play  optimally  throughout, 
the  terminal  conditions  for  the  game  will  be  met.  Evalua- 
tion of  F requires  integration  of  the  system  dynamics  using 
some  present  estimate  of  the  X(t^)  for  a period  (t-  t ), 
given  a current  estimate  of  the  final  time  t ^ . Now  consider 


94 


the  minimization  of  the  scalar  cost  function 


min  FT(x)F(x)  » J* 
x 

The  first  order  necessary  conditions  are 


Ml  « o = 2FTF 
3x  — x 


the  solution  of  which  is  either 


(B-3) 


(B-4) 


(B-5) 


Another  solution  exists  when  Fx  is  singular  for  some  F ^ 0. 
Using  these  three  possible  solutions  together  with  Eq  (B-3) 
the  three  solutions  can  be  separately  identified  as  follows 


(a) 

FT(x) F (x)  < e 

(B-6) 

(b) 

T T 

F*F  > kF 1 F 

X 

(B-7) 

(c) 

lpxl  = o 

(B-8) 

In  (a)  the  solution  of  Eq  (B-2)  has  been  achieved  to 

within  e.  In  (b)  the  gradient  of  F,  Fx  is  such  as  to  predict 

a very  large  step  d in  a Newton-Raphson  search  for  the  minima 
T 

of  F F.  The  constant  k allows  some  control  over  what  is  and 
what  is  not  an  acceptably  large  step.  Normally  in  gradient 
search  techniques  a singular  matrix  Fx  precludes  finding  the 
gradient  direction  unless  the  generalized  inverse  of  Fx  is 
used . 


A Fortran 


subroutine  published  at  Ref  10  searches  for 

• V 

a solution  to  Eq  (B-2)  until  F*F  is  less  than  some  e,  w ich 
is  part  of  the  calling  parameters.  It  initially  estimates 
by  numerical  differentiation  using  a user  supplied  step 
size  and  after  taking  the  inverse  of  F__  thereafter  uses 
approximations  of  * based  on  the  steps  it  takes  in  the 
search.  Should  the  gradient  scheme  define  an  unacceptably 
large  step,  as  controlled  by  k in  Eq  (B-7),  then  control  is 
returned  to  the  calling  program.  By  continuously  approxi- 
mating ^ then  the  problems  associated  with  Eq  (8-8)  are 
avoid  ed . 

Since  the  scheme  uses  an  approximation  to  * it  tends 
to  take  cautious  steps  in  the  search  for  a solution  so  that 
the  assumptions  in  making  the  appr ox im at  ion  axe  not  violated. 
Changes  in  this  step  size  arc  made  if  the  predicted  value 
of  F(x+6)  and  the  actual  value  arc  either  very  cioce  in 
which  case  A the  step  size  is  increased  or  wildly  different 
in  which  case  A is  reduced.  In  the  algorithm  the  Newton- 
Raphson  step  is  calculated  and  if  this  is  less  than  or  equal 
to  A it  is  used  directly.  If  this  is  not  the  case  the  algo- 
rithm predicts  the  optimum  step  sire  in  the  direction  of 
steepest  descent  assuming  that  it  is  searching  on  a quadratic 
surface.  This  step  size  is  compared  with  A as  before. 

Should  neither  steepest  descent  nor  Newt on - Raph son  predict 
a step  size  less  than  A a combination  is  used  to  calculate 
a s > ♦*!>  of  A in  a direct  ion  somewhere  between  the  two. 


Because  the  scheme  has  a pseudo  cost  function  F*F 

which  has  a minima  at  the  solution  of  F(x)  - 0,  it  retains 

T 

the  value  of  x that  has  produced  the  smallest  value  of  F F 
and  on  an  error  return  to  the  calling  program  usually  re- 
turns this  "best"  value  of  x no  matter  how  far  away  the 
current  estimate  of  x may  be. 

This  then  is  the  algorithm  that  has  been  used  through- 
out to  solve  the  two  point  boundary  value  problems  that 
represent  the  saddle  point  solution. 


Numerical  Problems 

The  Hamiltonian  in  the  game  formulation  was  independent 
of  time  and  hence 


dH 

dt 


3HT 

dx 

+ iy 

T 

dX 

3H 

3 X 

dt 

3A 

dt 

3 1 

3HT 

f + 

fT 

d A 

3 H 

3 X 

X T 

dt 

T 

3 1 

-XTf  ♦ 

fT; 

0 

(B-9) 

The  Hamiltonian  at  final  time  was  zero  and  hence  should  have 
remained  so.  When  a set  of  states,  representing  the  termina- 
tion of  a differential  game  with  a miss  of  100  feet  in  each 
direction,  was  integrated  backwards  in  time,  the  Hamiltonian 
oscillated  rapidly  and  then  gradually  settled  at  some  value 
The  differential  game  was  coded  in  an  unnormalized  form, 
that  is  positions  in  feet,  velocities  in  feet  per  second, 
etc.  Even  when  the  integration  step  size  of  5 ysec  was  used 
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and  the  Hamiltonian  oscillated,  but  to  much  smaller  limits 
than  previously,  the  trouble  seemed  to  lie  in  the  integra- 
tion and  not  the  equations  being  integrated.  The  integration 
routine  used  was  a 4th  order  Runge-Kutta  to  start  and  there- 
after a 4th  order  pred ic tor -correct  or . The  choice  of  fixed 
or  variable  step  size  could  be  made  in  the  calling  parameters. 
For  a variable  step  size  the  routine  would  increase  or  de- 
crease the  step  size  as  necessary  to  keep  the  error  bound 
below  some  given  value.  As  the  step  size  required  changing 
the  scheme  would  use  Runge-Kutta  to  start  afresh  before 
moving  to  predictor-corrector  integration.  It  was  noticed 
that  it  was  always  the  Runge-Kutta  steps  that  produced  the 
oscillation  which  would  then  settle  down  considerably  during 
predictor-corrector  integration.  When,  in  variable  step 
size  mode,  the  Runge-Kutta  scheme  was  used  to  start  afresh 
after  each  change  of  step  size,  back  came  the  oscillations. 

With  a miss  distance  of  some  176  feet  the  values  of  A (t,.) 

Y f 

and  ^CJ(tp)  would  lie  between  +10  and  -10  but  their  slopes 

would  be  of  the  order  1.E5.  Since  the  states  and  costates 

were  being  integrated  by  a single  routine  the  range  of 

slopes  that  the  one  routine  had  to  handle  was  from  -1E5  to 

+1E5.  The  value  of  X was  iargely  aerermined  by  such  terms 

as  X -v  where  A was  200  and  v was  800. 
y.  x 

The  expression  for  X contained  X , X and  X . Both 
r y y v o 

X and  A contained  terms  in  A so  that 
t-j  V y 
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(B-10) 


X * AX  + BX  + CX  + ... 
y Y <J  v 

X - DX  + . . . (B-ll) 

ay  v 1 

X = EX  + . . . (B- 12) 

V Y 

and  thus 

A = (A2+BD+CE)X^  + ...  (B- 13) 

and  further  differentiation  would  not  separate  the  costates. 
However  it  does  show  that  an  exponential  form  of  the  homo- 
geneous solution  could  contribute  the  stiffness  seen  in 
integrating  the  costate  equations. 

That  the  trouble  was  largely  independent  of  the  large 
value  of  the  slopes  could  be  seen  later  in  the  integration 
where  the  slopes  were  still  in  the  range  1E5  but  the 
Hamiltonian  had  more  or  less  stabilized  and  despite  the  error 
generated  initially  was  stable  to  3 or  4 significant  digits. 

By  splitting  the  integration  of  the  state  equations 
into  two  phases  good  results  were  achieved  and  if  the  inte- 
gration procedure  was  reversed  the  original  states  could  be 
returned  to, within  about  6 or  7 significant  digits  on  a 
IS  digit  machine.  The  total  time  was  divided  into  256  steps 
and  the  last  6 steps  were  further  subdivided  into  200. 

This  gave  a very  fine  step  size  around  the  final  time 
and  an  acceptably  coarse  step  size  away  from  t ^ which  kept 
the  time  spent  on  integration  as  small  as  possible  without 
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sacrificing  accuracy.  When  integrating  forward  into  the 
points  in  time  at  which  Runge-Kutta  steps  were  taken  would 
be  different  from  the  points  when  integrating  away  from  t^ 
thus  causing  differences.  The  values  of  the  states  and 
costates  achieved  by  integrating  away  from  t ^ were  therefore 
used  as  an  initial  guess  for  the  two  point  boundary  value 
problem,  and  when  subroutine  NS01A  had  reduced  the  errors 
in  the  transver sality  conditions  to  some  prespecified  level 
it  was  found  that  the  terminal  state  had  changed  slightly. 

For  the  situation  where  the  missile  hit  the  aircraft 
the  values  of  the  terms  A,  B,  C,  D and  E in  Eq  (B-13)  were 
such  that  the  Hamiltonian  remained  at  2.E-14  throughout  the 
integration . 
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